Skip to content

mongodb: fix leaked changestream cursor#4565

Open
jgao54 wants to merge 1 commit into
mainfrom
mongo-too-many-cursor
Open

mongodb: fix leaked changestream cursor#4565
jgao54 wants to merge 1 commit into
mainfrom
mongo-too-many-cursor

Conversation

@jgao54

@jgao54 jgao54 commented Jul 10, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor

Fixes a changestream cursor resource leak that can get triggered when the context is already cancelled. ChangeStream.Close executes a killCursors command against the server; but skips it if context is invalid. This has led to the error "Cannot open a new cursor since too many cursors are already opened" for DocumentDB instances that cap cursor limit. Fix is to always use a fresh context.

Also fixes leak caused by ignoring reassignment of changestream when recreateChangeStream is called. Without the closure, defer captures the original changestream at time it was evaluated.

Fixes: DBI-902

@jgao54 jgao54 requested a review from a team as a code owner July 10, 2026 19:21
@claude

claude Bot commented Jul 10, 2026

Copy link
Copy Markdown

Code review

No issues found. Checked for bugs and CLAUDE.md compliance.

@jgao54 jgao54 force-pushed the mongo-too-many-cursor branch from 4108d6a to 3b94f46 Compare July 10, 2026 19:27
@codecov

codecov Bot commented Jul 10, 2026

Copy link
Copy Markdown

❌ 2 Tests Failed:

Tests completed Failed Passed Skipped
3021 2 3019 391
View the top 2 failed test(s) by shortest run time 
github.com/PeerDB-io/peerdb/flow/e2e::TestPeerFlowE2ETestSuiteMySQL_CH
Stack Traces | 0.02s run time 
=== RUN   TestPeerFlowE2ETestSuiteMySQL_CH
=== PAUSE TestPeerFlowE2ETestSuiteMySQL_CH
=== CONT  TestPeerFlowE2ETestSuiteMySQL_CH
2026/07/10 19:26:21 INFO Received AWS credentials from peer for connector: ci x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN}
--- FAIL: TestPeerFlowE2ETestSuiteMySQL_CH (0.02s)
2026/07/10 19:39:59 INFO Received AWS credentials from peer for connector: ci x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN}
2026/07/10 19:39:59 INFO Received AWS credentials from peer for connector: clickhouse x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN}

github.com/PeerDB-io/peerdb/flow/e2e::TestPeerFlowE2ETestSuiteMySQL_CH/Test_MySQL_BinlogIncident
Stack Traces | 58s run time 
=== RUN   TestPeerFlowE2ETestSuiteMySQL_CH/Test_MySQL_BinlogIncident
=== PAUSE TestPeerFlowE2ETestSuiteMySQL_CH/Test_MySQL_BinlogIncident
=== CONT  TestPeerFlowE2ETestSuiteMySQL_CH/Test_MySQL_BinlogIncident
2026/07/10 19:39:01 INFO Received AWS credentials from peer for connector: ci x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN}
2026/07/10 19:39:01 INFO Received AWS credentials from peer for connector: clickhouse x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN}
2026/07/10 19:39:01 INFO fetched schema x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} table=e2e_test_mych_8nq80ex6.test_nullengine
    clickhouse_mysql_test.go:2157: 
        	Error Trace:	.../flow/e2e/mysql.go:130
        	            				.../flow/e2e/clickhouse_mysql_test.go:2157
        	            				.../hostedtoolcache/go/1.26.4.../src/runtime/asm_amd64.s:1771
        	Error:      	Received unexpected error:
        	            	MySQL execute error: readInitialHandshake: io.ReadFull(header) failed. err EOF: connection was bad
        	Test:       	TestPeerFlowE2ETestSuiteMySQL_CH/Test_MySQL_BinlogIncident
2026/07/10 19:39:29 INFO fetched schema x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} table=e2e_test_mych_5o3mc7sx.test_unsigned
--- FAIL: TestPeerFlowE2ETestSuiteMySQL_CH/Test_MySQL_BinlogIncident (57.98s)

To view more test analytics, go to the Test Analytics Dashboard
📋 Got 3 mins? Take this short survey to help us improve Test Analytics.

@github-actions

github-actions Bot commented Jul 10, 2026

Copy link
Copy Markdown
Contributor

🔄 Flaky Test Detected

Analysis: Test_MySQL_BinlogIncident failed on a transient MySQL handshake EOF ("readInitialHandshake: io.ReadFull(header) failed. err EOF: connection was bad") while connecting to its own throwaway debug MySQL container during setup — a connection/startup-timing race, not a logic bug.
Confidence: 0.9

✅ Automatically retrying the workflow

View workflow run

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@jgao54